IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, 
ELECTRONIC COMMERCE SYSTEM, ELECTRONIC COMMERCE 
METHOD AND A COMPUTER PROGRAM PRODUCT 



BACKGROUND OF THE INVENTION 

Field Of The Invention 

This invention relates to an image 
processing apparatus, an image processing method, 
electronic commerce system, electronic commerce 
method and a computer program product, and 
especially relates to a technology for prohibiting 
counterfeiting of a bill or an image by attaching 
additional information in digital image data to be 
printed. The invention further relates to how to 
use user information obtained in sales over 
electronic commerce of an article of commerce by 
using network. 



Description Of The Related Art 

Conventionally, an image processing 
apparatus scans - an original image and records the 
original image faithfully on a paper or an 
electrical recording medium. Meanwhile, it is 
5 possible to reproduce an image to be printed 

faithfully on paper by recent image processing 
technology in accordance with the progress of color 
process technology. With this technology, an image 
processing apparatus can copy and print forgeries of 

10 objects, like bills or valuable securities, for 

abuse or mischief. 

For a solution of the above-mentioned 
problem, it is proposed to multiplex information of 
a name of maker or an image processing apparatus or 

15 a manufacture number of an image processing 

apparatus to image data to be printed by the image 
processing apparatus and to identify the image 
processing apparatus by using the multiplexed 
information . 

2 0 The proposed method using the multiplexed 

information is preferable when applied to a 
comparatively expensive image processing apparatus, 
like a color copying apparatus. However, it is not 
so preferable when applied to an inexpensive image 

25 processing apparatus, like a color printer being 

popular recently. Since the expensive image 
processing apparatus is thrown out rarely and 
information of an owner of the expensive image 
processing apparatus is surely retained for after- 

30 purchase servicing, it is possible to specify an 
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owner of the expensive image processing apparatus 
used in a forgery as significant evidence in case 
that a manufacture number of the expensive image 
processing apparatus is detected from a 
counterfeited bill or valuable security. 
5 On the other hand, an inexpensive image 

processing apparatus is easy to throw out . So a 
forger is likely to conceal his efforts by throwing 
out an inexpensive image processing apparatus and 
destroy evidence when a manufacture number of the 
10 forger's image processing apparatus is specified. 

SUMMARY OF THE INVENTION 

An object of the present invention is to 
15 provide solutions for the above problems. 

Another object of this invention is to 
decrease the likelihood that an image processing 
apparatus is used to make unauthorized images by 
setting information to specify a seller or a buyer 
2 0 of the image processing apparatus inside it. As a 

result, if the output image were an unlawful image, 
the seller or the buyer could be specified from the 
user information multiplexed the output image. 

One particular object of the present 
25 invention is to provide an image processing 

apparatus which has user information storing means 
for storing user information related to a seller or 
a buyer of the image processing apparatus; and 
multiplexing means for multiplexing the user 



information to an output image formed by the image 
processing apparatus . 

Another object of the present invention is 
to provide an electronic commerce system which 
performs through a network system connecting one 
terminal with another terminal through a network, 
the electronic commerce system comprising 
information setting means for setting information 
related to a seller or a buyer obtained through the 
network, and multiplexing means for multiplexing 
said information set by the information setting 
means to information of an output image. 

Another object of the present invention is 
to provide a computer program product, comprising a 
computer readable medium having computer program 
codes for executing image processing on an image 
processing apparatus, said product including: user 
information storing procedure codes for storing user 
information related to a seller or a buyer of the 
image processing apparatus; and multiplexing 
procedure codes for multiplexing the user 
information to a output image. 

Another object of the present invention is 
to provide a computer program product, comprising a 
computer readable medium having computer program 
codes for an electronic commerce system which 
performs through a network system connecting one 
terminal with another terminal through a network, 
the computer program product comprising information 
setting procedure codes for setting information 
related to a seller or a buyer obtained through the 



network, and multiplexing procedure codes for 
multiplexing said information set by the information 
setting means to information of an article for sale. 

It is an important advantage of the 
present invention that the present invention can 
offer information related to a seller or a buyer and 
an article for sale. So if the article were used 
unlawful, it would be easy to specify the seller or 
the buyer. 

Still another object of the present 
invention is to provide an image processing 
apparatus and an electronic commerce system and a 
computer program product that have a new function 
not known before. 

The foregoing and still other objects, 
features and advantages of the present invention 
will become fully apparent from the following 
description to be taken in conjunction with the 
attached drawings . 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram which shows a 
public key certification. 

FIG. 2 is an outline of a system of the 1 st 
and the 2 nd embodiment of the present invention. 

FIG. 3 is a flow chart that shows an 
operation of the 1 st embodiment of the present 
invention . 



FIG. 4 is a flow chart that shows an 
operation of the 2 nd embodiment of the present 
invention . 

FIG. 5 is an example of structure that 
shows an image processing apparatus of the 
embodiments of the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[First embodiment] 

Recently, electronic commerce (Web-based 
commerce) to deal with an article for sale on a 
network is widely used in accordance with 
development of a network, like the Internet. A 
procedure of the electronic commerce is the 
following. A virtual mall is opened on the network, 
an article for sale like a printer is exhibited on 
the virtual mall, a user makes a contract for 
purchasing an article selected by the user, the 
article selected by the user is delivered, and cost 
of the article selected by the user is paid by a 
credit card. 

Along with the popularization of the 
electronic commerce, Public Cryptograph Technology 

(cf. "modern cryptograph theory" Institute of 
Electronics, Information and Communication 
Engineers) was proposed to certify a seller and a 
buyer and to prevent failure of performance after a 
contract was done. Infrastructure using the public 
key is called PKI (Public Key Infrastructure) and 
standardization using X.509 public key certification 



(see Fig. 1) is founding in IETF (Internet 
Engineering Task Force: International Organization 
for Standardization) . 

Public key certification is data which 
contains an identification name (name to specify 
each person) published from a trusted third-party 
body called a "Certification Authority" and each 
person's public key is signed by a Certification 
Authority. It is possible to prohibit altering 
content of the public key by Certification 
Authority' s signature and a user received the publi 
key certification can verify that the public key 
contained in the public key certification is owned 
person of an identification name of the public key 
certification under Certification Authority's 
guarantee. In short, PKI (Public Key 
Infrastructure) provides a mechanism to bind a 
person (or a server etc.) to his or her public key 
with certainty. It is frequently used in an 
electronic credit system, to certify a seller and a 
buyer . 

The following is an example of a method 
which decreases the likelihood of counterfeiting 
bills or valuable securities by using above- 
mentioned electronic commerce. 

FIG. 2 is a block diagram which shows an 
example of such a system, applying this invention 
for an image processing apparatus, to prevent 
counterfeiting. Reference numeral 200, in FIG. 2, 
is a network to connect a seller with a buyer, 
server 201 is a seller's server operating a virtual 



mall etc., personal computer 202 is a personal 
computer for a buyer to purchase an article for 
sale, printer 203 is a printer which has already 
been purchased by using electronic commerce, printer 
204 is a printer sold over the counter sale, CD-ROM 
2 05 is a recording medium storing a driver and 
attached printer 203, 204. 

FIG. 3 indicates an example of a 
processing for purchasing an article for sale. 

When an owner (buyer) of personal computer 
202 decides to purchase an article of sale sold in a 
virtual mall having server 201 (step 301) , a 
contract process, which is agreement about purchase 
including, for example, a method of payment, a 
method of transportation, and conditions of use, is 
made through network 2 00 (step 3 02) . In case the 
buyer makes use of the above-mentioned electronic 
credit system like electronic money for payment, the 
public key certification can be used as important 
information to specify the buyer. In case the buyer 
makes use of a method of transferring money from the 
buyer's account to the seller's account or a credit 
card for payment, the buyer's account number or 
information of the credit card can be used as 
important information to specify the buyer. The 
buyer inputs such information (the public key 
certification, the account number or information of 
the credit card) by personal computer 2 02 when the 
buyer makes his contract. In this embodiment, since 
the article for sale is an image processing 
apparatus like a printer which can be used 



counterfeiting bills or valuable securities, the 
seller having server 201 sets the information to 
specify the buyer to a specific portion of the image 
processing apparatus before the seller delivers it 
to the buyer (step 3 03) . 

Then the seller delivers the image 
processing apparatus (step 304) . 

The specific portion of the image 
processing apparatus may be hardware like ROM (Read 
Only Memory) or a part of software like a printer 
driver. It is important that the image processing 
apparatus is set with the information (the public 
key certification, the account number or information 
of the credit card) to specify the buyer instead of 
a manufacture number of the image processing 
apparatus, and the information to specify the buyer 
is multiplexed in a print image when the print image 
is output by the image processing apparatus. 

In case a forgery of a bill or valuable 
security is found later, and even if the image 
processing apparatus which printed the forgery has 
been thrown out, it is possible to identify an 
unlawful owner by using the information to specify 
the buyer multiplexed to the forgery.' The public 
key certification, the account number or information 
of the credit card is better than the manufacture 
number of an- image processing apparatus, since it is 
necessary to confirm personal information for 
obtaining the public key certification, the account 
number or information of the credit card. 
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The seller having server 2 01 or a bureau 
of administrative services managing articles for 
sale manages the information (the public key 
certification, the account number or the information 
of the credit card) by using a database instead of 
multiplexing the information to specify the buyer to 
a print image directly and an image processing 
apparatus may be set with information to access the 
information to specify the buyer on the database. 
In this case, it is possible for the seller or the 
bureau of administrative services to get the 
information to specify buyer on the database from 
the information to access the data. Under these 
circumstances, the information to specify the buyer 
in step 3 02 changes to the information to access the 
data on the database. 

When it is impossible to set the 
information of a buyer to an image processing 
apparatus in light of privacy protection, it is 
effective to set information related to sale like 
seller or date of sale to the image processing 
apparatus for preventing forgery of bills or 
valuable securities. Then, the information to 
specify the buyer in step 302 changes to the 
information related to sale or date of sale. 



[Second embodiment ] 

Whereas an article of sale is purchased by 
using electronic commerce in a first embodiment, in 
this embodiment, setting information to specify a 
user for prohibiting counterfeiting bills or 
valuable securities is performed for an article 
having purchased or an article purchased over the 
counter sale. 

The structure of this embodiment's system 
looks like FIG. 2. Printer 203 is a printer that has 
already purchased, printer 204 is a printer sold 
over the counter sale, CD-ROM 205 is a recording 
medium storing a driver and attached printer 203, 
204 . 

FIG. 4 indicates an example of a process 
for an article purchased not using a network. 

At first, the following is an example of a 
process for an article that has already been 
purchased . 

Upgrade process of a driver by a printer 
is usually performed several times by download 
through a network like Internet. A manufacturer 
informs users of notice of upgrade by a direct mail, 
an e-mail or home page through a network. When a 
user of personal computer 202 upgrades a driver 

(step 401) , the user designates a predetermined 
process and the driver is downloaded from server 2 01 

(step 4 03) . The download is performed after 
inputting information to specify the user of printer 
204 from personal computer 202. It goes without 
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saying that the information is utilized for 
multiplexing like that of the first embodiment. 

The next case is a process for a printer 
sold over the counter sale (printer 204) . CD-ROM 
205 attached to printer 204 has recorded on it, at 
least, information of a store selling printer 204. 
When a user of printer 204 installs a driver by- 
using CD-ROM 2 05, the information of a store is set 
to a specific portion of printer 204. It goes 
without saying that the information is utilized for 
multiplexing like that of the first embodiment. 

When a user of printer 204 installs a 
driver by using CD-ROM 2 05, personal computer 2 02 
may connect to server 201 automatically under 
controlled a program of CD-ROM 2 05 and personal 
computer 202 instead of installing the printer 
driver from CD-ROM 205. Then information to specify 
the user of printer 2 04 is input by the user. 
Personal computer 2 02 downloads the last printer 
driver and an optional function (program) from 
server 2 01 in accordance with the input by the user 
(step 403) . The input information is set to a 
specific portion of printer 204 or the printer 
driver downloaded from server 201 under control of 
the program of CD-ROM 205 and personal computer 202. 
The specific portion of printer 204 may be hardware 
like ROM (Read Only Memory) . 

Printer 204 can multiplex the information 
to specify the user of printer 204 to an output 
image, like that of the first embodiment, under the 
above-mentioned procedure. As a result, when a 
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forgery is found, it is possible to identify the 
forger with certainty based on detecting the 
information multiplexed to the forgery. 

In case a user of server 201 manages users 
of articles by using a database, the information 
multiplexed to the forgery is preferable to 
information to access the data on the database. If 
a user does not download any data from server 210, 
stored information prerecorded on CD-ROM 2 05 is 
substituting for the information to specify user. 

[Third embodiment] 

One well-known multiplexing method is a 
so-called digital watermark, and a digital watermark 
is used in the following to function as a 
multiplexing method for the information to specify a 
user in the first and second embodiment . 

Digital watermark technology is a 
technology that adds copyright information or user 
information or various identification numbers in an 
image by converting pixel value of digital data of 
the image in secret. It is almost impossible to 
distinguish an original from an image having an 
added the digital watermark with the naked eye. 
However, the hidden copyright information etc. as 
the digital watermark can be displayed by using 
software to detect the digital watermark. It is 
possible to obtain the copyright information or the 
user information or identification numbers by 
detecting this digital watermark from image data and 
to identify an illegal copy. First, added 
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information by digital watermark must be non- 
visible, in other words the information must be 
added without degradation of an original. Second, 
the information must keep on remaining in an 
original, in other words the information must not be 
lost by a retouch or an attack like a data 
compressing process, a filter process etc. Third, 
an amount of added information must be able to be 
variable. As stated above, digital watermarks need 
quality, tolerance and flexible amount of 
information . 

The following method is known as a method 
for adding digital watermark. A method of adding 
digital watermark can be roughly categorized by two 
methods: adding information in a spatial area, and a 
frequency area for a mult i -valued still image. 

For example, "Techniques for Data Hiding", 
Proceeding of The SPIE, San Jose CA , USA, (IBM) W. 
Bender, D. Gruhl , N. Morimoto, February 1995, and 
U.S. Patent 5,636,292 are examples of methods adding 
information in a spatial area. 

The following gives a few examples which 
are methods adding information in a frequency area. 
A method using "Discrete Cosine Transform" is a 
digital watermark method in a frequency area for 
protecting copyright of a digital image (NTT) 
Nakamura, Ogawa, Takashima, SCIS '97-26A, Jan 1997. 
A method using "Discrete Fourier Transform" is a 
watermark signature method for an image by using PN 
series (National Defense Academy) Ohonishi, Oka, 
Matui, SCIS '97-26B, Jan 1997. Methods using 
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"Wavelet Transform" are experimental considerations 
related to safety and reliability of digital 
watermark technology using wavelet transform 
(Mitsubish, Kyushu university) Ishizuka, Ishi, 
Sakurai, SCIS '972d, Jan 1997 and digital watermark 
image compression based on wavelet transform, about 
robustness for the conversion process (Matushita) 
Inoue, Myazaki, Yamamoto, Katura, SCIS '98-32A, Jan 



information in a spatial area is low degradation and 
low tolerance and a method adding information in a 
frequency area is comparatively high degradation and 
high tolerance. It is possible to select a 
preferable method for each case. 

[Fourth embodiment] 

The above-mentioned process is not only 
useful for the image processing apparatus but also 
general commodities. In other words, it is 
effective to set information to specify a buyer or a 
seller of an article for sale which is purchased by 
using electronic commerce and to output information 
related to the article for sale as prints or signals 
through a network. 



buyer or a seller of an article for sale (a personal 
computer or a scanner or an image processing 
software) and to send the set information to an 
image processing apparatus as a final output 



1998 . 



Generally speaking, a method adding 



For example, it. is possible to set an 
information to be gotten on purchase to specify a 
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apparatus and to multiplex the information to a 
printed image by the image processing apparatus. 
The information to specify a buyer or a seller of an 
article for sale may be multiplexed to data to be 
sent through a network by using the digital 
5 watermark, except the printed image. 

FIG. 5 is an example of structure that 
shows an image processing apparatus of the 
embodiments of the present invention. Image 
processing apparatus 50 is a personal computer, a 

10 printer or a system including a personal computer 

and a printer. Interface 51 is a parallel 
interface, a serial interface like Universal serial 
bus, IEEE1394, or wireless interface. User 
information storing unit 52 is a random access 

15 memory, a hard disc ora detachable memory. Article 

information for sale storing unit 53 is random 
access memory, a hard disc, CD-ROM, memory card like 
compact flash. Image information storing unit 54 is 
random access memory, a hard disc, CD-ROM, memory 

20 card like compact flash. Multiplexing unit 55 is a 

printer controller forming a print image or a 
printer engine developing a print image . 
Information setting unit 56 is a central processor 
unit of image processing apparatus 50. The 

25 following processing is performed under control of 

the central processor unit. 

Interface 51 is utilized for communicating 
image processing apparatus 50 to an external 
apparatus. User information storing unit 52 is a 

3 0 storing means for storing user information related 
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to a buyer or a seller of the image processing 
apparatus 50. Article information for sale storing 
unit 53 is a storing means for storing information, 
input through a network, related to a buyer or a 
seller of an article of sale. Image information 
storing unit 54 stores printed image information. 
Information setting unit 56 sets predetermined 
information related a buyer or a seller of image 
processing apparatus 50, and multiplexing means 55 
multiplexes the predetermined information to a 
printed image when the printing image is output . 

The object of the present invention can be 
also achieved by providing a storage medium storing 
program codes for performing the aforesaid processes 
to a system or an apparatus, reading the program 
codes with a computer (e.g., CPU, MPU) of the system 
or apparatus from the storage medium, then executing 
the program. 

In this case, the program codes read from 
the storage medium realize the functions according 
to the embodiments, and the storage medium storing 
the program codes constitutes the invention. 

Further, the storage medium, such as a 
floppy disk, a hard disk, an optical disk, a 
magneto-optical disk, CD-ROM, CD-R, a magnetic tape, 
a non-volatile type memory card, and ROM can be used 
for providing the program codes. 

Furthermore, besides aforesaid functions 
according to the above embodiments are realized by 
executing the program codes which are read by a 
computer, the present invention includes a case 
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where an OS (Operating System) or the like working 
on the computer performs a part or entire processes 
in accordance with designations of the program codes 
and realizes functions according to the above 
embodiments . 

Further, the present invention includes a 
case where CPU equipped with an extender board or 
unit performs a part or entire processes in 
accordance with designations of the program codes 
after the program codes is stored on a memory 
connected the extender board or unit of a computer 
and realizes functions according to the above 
embodiments . 

Further, according to the above-mentioned 
embodiment, since a purchased image processing 
apparatus can be set with information to specify a 
buyer or a seller of the purchased image processing 
apparatus by using user information related to 
electronic commerce, it is possible to get 
information to specify an owner from a forgery of 
bills or valuable securities. As a result, even 
though it is possible to perform an unlawful image 
processing by using the image processing apparatus, 
it is easy to identify an unlawful owner even in 
case the image processing apparatus has been thrown 
out . 

Further, since information related to a 
buyer or a seller of an article of sale input 
through a network is multiplexed an output image, an 
unlawful process by using the article of sale 
obtained by electric commerce can be prohibited. 
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Although the invention has been described 
in its preferred form with a certain degree of 
particularity, it is understood that the present 
disclosure of the preferred form has been changed in 
the details of construction and the combination and 
the arrangement of parts may be resorted to without 
departing from the spirit and scope of the invention 
as hereinafter claimed. 



